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Qb^ Abstract 

(-H Toral introduced so-called cooperative Parrondo games, in which there 

■ *— > are N > 3 players arranged in a circle. At each turn one player is randomly 

^ chosen to play. He plays either game A or game B, depending on the 

S strategy. Game A results in a win or loss of one unit based on the toss 

of a fair coin. Game B results in a win or loss of one unit based on the 

, toss of a biased coin, with the amount of the bias depending on whether 

►^ none, one, or two of the player's two nearest neighbors have won their 

QQ most recent games. Game A is fair, so the games are said to exhibit 

^-H the Parrondo effect if game B is losing or fair and the random mixture 

00 C := (1/2) (A -I- B) is winning. With fig (resp., fi^) denoting the mean 

^^ profit per turn to the ensemble of A'^ players always playing game B (resp., 

f^ C), we give sufficient conditions for limjv_>oo Ms to exist and show that 

^> limAr_>cx> Mc nearly always exists, with the limits expressible in terms of 

rvj a parameterized spin system on the one-dimensional integer lattice. For 

^-H a particular choice of the parameters, we show that the Parrondo effect 

JL" (i.e., fiB 5- and /i^ > 0) is present in the A-player model if and only if 

.5^ A is even. For the same choice of the parameters, we show that, with a 

S^ suitable interpretation and for certain initial distributions, the Parrondo 

$_j effect is present in the spin system if and only if N is even, N being the 

number of consecutive players whose collective profit is tracked. 
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1 Introduction 

In Toral's (2001) cooperative Parrondo games, there are A^ > 3 players labeled 
from 1 to iV and arranged in a circle in clockwise order. At each turn, one player 
is chosen at random to play. Call him player i. He plays either game A or game 
B, depending on the strategy. In game A he tosses a fair coin. In game B he 
tosses a po-coin (i.e., po is the probability of heads) if his neighbors z — 1 and 
i + 1 are both losers, a pi-coin if i — 1 is a loser and i + 1 is a winner, a p2-coin 
if i — 1 is a winner and i + 1 is a loser, and a p^-cova if i — 1 and i + 1 are both 
winners. (Because of the circular arrangement, player is player N and player 
A'^ + 1 is player 1.) A player's status as winner or loser depends on the result of 
his most recent game. Of course, the player of either game wins one unit with 
heads and loses one unit with tails. Under these assumptions, the model has an 
integer parameter N > 3 and four probability parameters Po,Pi,P2,P3 S [0, 1]. 
(The model described in the abstract is the special case in which pi = P2-) 
Game A is fair, so the games are said to exhibit the Parrondo effect if game B 
is losing or fair and the random mixture C := {1/2){A + B) (i.e., toss a fair coin 
to determine which game to play) is winning. Toral used simulation to find a 
case in which the Parrondo effect appears, thereby providing a new example of 
Parrondo's paradox (Harmer and Abbott 2002, Abbott 2010). 

Extending the work of Mihailovic and Rajkovic (2003) (see also Xie et al. 
2011), Ethier and Lee (2012) showed how to compute ^g (resp., /^p), the mean 
profit per turn to the ensemble of N players always playing game B (resp., C), 
for 3 < A^ < 19. Their numerical results suggested that these means converge as 
N —>■ oo, and that the Parrondo effect (i.e., fig < and Hq > 0) is present in the 
limit for a set of parameter vectors having nonzero volume. In the present paper 
we give sufficient conditions for lim7v_^Qo /i^ to exist and show that limjv-i-oo Mc 
nearly always exists. Further, we show that the limiting values are expressible 
in terms of a parameterized interacting particle system, or spin system, on the 
one-dimensional integer lattice Z. For a particular choice of the parameters, 
namely po = 1, pi = P2 G (1/2,1), and p^ — 0, we show that the Parrondo 
effect is present in the A^-player model if and only if N is even. For the same 
choice of the parameters, we show that, with a suitable interpretation and for 
certain initial distributions, the Parrondo effect is present in the spin system 
if and only if N is even, N being the number of consecutive players whose 
collective profit is tracked. 

Section [2] describes the iV-player model and the associated discrete-time 
Markov chain. Section Is] establishes a strong law of large numbers (SLLN) for 
the sequence of profits to the ensemble of N players playing game B, giving 
several formulas for the mean parameter. Section l4] treats the special case in 
which we can confirm the Parrondo effect for all even N > A. Section[5]describes 
what one might mean by Parrondo's paradox in continuous time, and establishes 
an SLLN for the sequence of profits to the ensemble of N players playing game 
B, assuming a continuous-time Markov chain model. Section [6] introduces the 
related spin system and derives its basic properties, giving sufficient conditions 
for ergodicity. Section [7] establishes an SLLN for the sequence of profits to 



one of the infinitely many players playing game B, assuming the spin system 
model. Section [8] examines the special case of Section |4] in the context of the 
spin system. Section |9] concludes with a few open problems that we hope will 
be of interest to specialists in interacting particle systems. 

2 The discrete-time Markov chain 

Let us define the Markov chain, introduced by Mihailovic and Rajkovic (2003), 
that keeps track of the status (loser or winner, or 1) of each of the N players. 
It depends on an integer parameter A^ > 3 and four probability parameters 
Po,Pi,P2,P3 G [0, 1]. Its state space is the product space 



S := {a; = {xi,X2, ...,XN):x,e {0, 1} for i = 1, . . . , iV} = {0, 1}^ 

with 2^ states. Let mi{x) :— 2xi_i + Xi+i, or, in other words, mi{x) is the 
integer (0, 1, 2, or 3) whose binary representation is {xi^iXi+i)2', of course, 
Xq := xn and x^v+i '■— xi. Also, let x'' be the element of E equal to x except at 
the ith component. For example, x^ := (1 — xi,X2,X3, . . . ,xn)- The one-step 
transition matrix P for this Markov chain has the form 

(1) P(.,.'):^j^>=<^^ ''^'-'^ ..l,...,iV,.eE, 



(2) P{x,x):^N M ^ qmi(x) + JI Pmdx)], a; e E, 

where q™ := 1 — pm for vi — 0, 1, 2, 3 and empty sums are 0, and P{x, y) = 
otherwise. 

li Po,Pi,P2,P3 G (0, 1), then the Markov chain is irreducible and aperiodic, 
but this assumption is unnecessarily restrictive. Instead we will assume that 
pt),pi,P2,pz G [0, 1] and that the Markov chain is ergodic (i.e., there is a unique 
stationary distribution and the distribution at time n converges to it as n — ^ cx), 
regardless of the initial distribution). What exactly does this involve? The 
answer, which is not entirely intuitive, is provided by the following lemma. We 
denote the state comprising all Os by and the state comprising all Is by 1. 

Lemma 1. The Markov chain in S with one-step transition matrix P given by 
(Up and Q), where N >3 and po,pi,p2,P3 G [0, 1], has the following behavior. 

(a) If Po,P3 G (Ojl); then the chain is irreducible and aperiodic, with the 
following two exceptions. If N — 3 and pi — P2 — 0, then {000,001,010, 100} 
is closed, irreducible, and aperiodic, and the other four states are transient. If 
N = 3 and pi = p2 = 1, then {011,101,110,111} is closed, irreducible, and 
aperiodic, and the other four states are transient. 

(b) Suppose po = 1 and p^ G (0, 1). Then state is transient, and E — {0} 
is closed, irreducible, and aperiodic, with the following two exceptions. If pi = 



P2 = 1, then all states are transient except those without adjacent Os, and the 
set of such states is closed, irreducible, and aperiodic. If N is divisible by 3 and 
Pi = P2 = 0, then states 001 • • • 001, 010 • • • 010, and 100 • • • 100 are absorbing, 
and all other states are transient. 

(c) Suppose Pq = and p^ G [0,1). Then state is absorbing, and all 
other states are transient, with the following two exceptions. If N is divisible 
by 3, P3 = 0, and pi = P2 = 1; then there are four absorbing states, namely 

0, Oil • • • Oil, 101 • • • 101, and 110 • • • 110, and all other states are transient. If 
N — 3, P3 > 0, and pi — p2 ~ 1, then is absorbing, {011,101,110,111} is 
closed, irreducible, and aperiodic, and the other three states are transient. 

(d) Suppose Pq £ (0, 1) and ps = 0. Then state 1 is transient, and S — {1} 
is closed, irreducible, and aperiodic, with the following two exceptions. If pi = 
P2 ~ 0, then all states are transient except those without adjacent Is, and the 
set of such states is closed, irreducible, and aperiodic. If N is divisible by 3 and 
Pi — P2 — 1; then states Oil • • • Oil, 101 • • • 101, and 110 • • • 110 are absorbing, 
and all other states are transient. 

(e) Suppose po € (0,1] and ps = 1. Then state 1 is absorbing, and all 
other states are transient, with the following two exceptions. If N is divisible 
by 3, Po = 1, and pi = P2 = 0, then there are four absorbing states, namely 

1, 001 • • • 001, 010 • • • 010, and 100 • • • 100, and all other states are transient. If 
N = 3, Po < 1, and pi = J52 = 0, then 1 is absorbing, {000,001,010,100} is 
closed, irreducible, and aperiodic, and the other three states are transient. 

(f) Suppose Po ~ 1 and p^ = 0. If N is even, then states 01 • • • 01 and 
10 • • • 10 are absorbing, and all other states are transient, with two exceptions 
described below. If N is odd, then all states are transient except the 2N states 
in which Os and Is alternate with the single exception of a pair of adjacent Os 
or a pair of adjacent Is, and the set of such states is closed, irreducible, and 
aperiodic, with the same two exceptions, which are as follows. If pi = p2 ~ 0, 
then every state with no adjacent Is and no more than two consecutive Os is 
absorbing, and all other .states are transient. If pi = P2 == I7 then every state 
with no adjacent Os and no more than two consecutive Is is absorbing, and all 
other states are transient. 

(g) If Po ~ and p^ ~ 1, then states and 1 are absorbing, and all other 
states are transient. 

In summary, the Marltov chain is ergodic except when {po,P3) = (0,1); or 
when N is even and {po^Ps) = (IjO); or when {po,PitP2tP3) equals (1,0,0,0) 
or (1,1,1,0); or when N is divisible by 3 and either (pojPijP2) = (IjO, 0) or 
{pi,P2,P3) — (1,1,0); or when N — 3 and either {po,pi,p2) = (0,1,1) or 

(Pl,P2,P3) = (0,0,1). 

Furthermore, excluding the exceptions just listed, all of which involve non- 
uniqueness of stationary distributions, there is a closed, irreducible, aperiodic 
set of states, and all other .states are transient. Thus, uniqueness of stationary 
distributions and ergodicity are equivalent here. 

Proof. We prove part (f ) and leave the remainder of the proof as an exercise 
for the interested reader. Consider first the case of N even. Excluding the two 



exceptional cases, we notice that either (z) pi > and p2 < 1 or (m) pi < I and 
P2 > 0. Start with an arbitrary x. In case (i) we change the entries X2, ■ ■ ■ , xn 
from left to right as needed to get 01 • • • 01 if xi =0 and 10 • • • 10 if xi = 1. This 
requires 000 -^ 010, 001 ^ Oil, 111 ^ 101, and 110 ^ 100, all of which are 
legal moves under (i). In case (m) wc change the entries xi, . . . , x^v-i from right 
to left as needed to get 01 • • • 01 if xn — 1, 10 ••• 10 if a; at ~ 0. This requires 
000 -^ 010, 100 -^ 110, 111 -^ 101, and Oil -^ 001, ah of which are legal moves 
under (ii). 

We turn next to the case of N odd. Excluding the exceptional cases, again 
consider cases (i) and {ii) as above. Let us label the 2N states as follows. A 
state with alternating Os and Is except for a pair of adjacent Os is labeled by 
(0,z) if the first of the pair (in clockwise order) occurs at position i. A state 
with alternating Os and Is except for a pair of adjacent Is is labeled by (l,i) 
if the first 1 of the pair (in clockwise order) occurs at position i. Then the 
chain jumps from (0, i) to (1, « — 1) and (1, i + 1) with probabilities N^^p2 and 
N~^pi, respectively, and from (1, i) to (0, i — 1) and (0, i + 1) with probabilities 
N~^qi and N~^q2, respectively; as usual, 1 — 1 := TV and A^ + 1 := 1. In case 
(i), pi > and q2 > 0, so that the chain can cycle through the 2N states in 
clockwise order, whereas in case (ii), P2 > and qi > 0, so that the chain can 
cycle through the 2N states in counter-clockwise order. Irrcducibility of the set 
of 2N states leads to aperiodicity because P{x, a;) > 1 — 2/N > for every such 
X. To see that all other states are transient, we use the same argument as in the 
preceding paragraph. But here the target states are 01 • • • 010 and 10 • • • 101. 

Finally, regardless of the parity of A^, assume pi = P2 = 0. (The case 
Pi = P2 = I is symmetric.) Clearly, every state with no adjacent Is and no more 
than two consecutive Os is absorbing. Starting from an arbitrary x, flip each 
surrounded by Os, in clockwise order, say. Then Os occur only as singletons 
or pairs. Then do the same for Is, so that Is occur only as singletons or pairs. 
Since qi = q2 = 1, we can flip one of the Is in each adjacent pair. This may 
create three consecutive Os; if so, flip the center one. The resulting state has no 
adjacent Is and no more than two consecutive Os. D 

We have described what is known as game B. The description suggests 
that its long-term behavior should be invariant under rotation (and, if pi = p2, 
reflection) of the A^ players. 

Lemma 2. Let G be a subgroup of the symmetric group Sn. Let P be the one- 
step transition matrix for a Markov chain in S having a unique stationary distri- 
bution TT. For X — (xi, . . . , xtv) e S and a <E G, write x„ := (xcr(i), . . . , Xo-(7v))j 
and assume that P{x„,y„) — P{x,y) for all a E G and x,y G S. Then 
Ti{x„) = 7r(a;) for all a ^ G and a; G E. 

Proof. Given a Cz G, define the distribution tt^ on S by TTa{x) := tt{x^). Then 

aieE xGE xGE 

for all y G S, hence by the uniqueness of stationary distributions, n^ = n. D 



The lemma applies to our Markov chain (assuming it is ergodic) if G is the 
subgroup of cyclic permutations (or rotations) of (1,2,..., iV), that is, the group 
generated by 

(3) (a(l),a(2),...,a(iV)):=(2,3,...,7V,l). 

If pi = P2, then it also applies if G is the subgroup generated by (Is]) and the 
order-reversing permutation (or reflection) of (1, 2, ... , iV), 

((T(l),a(2),...,a(iV)):=(iV,iV-l,...,2,l). 

In this case G is known as the dihedral group of order 2N . The calculation that 
justifies these conclusions is 

(4) 



P[x„,{x'),)^P{x,,[x„Y' 


-^«) 




f^"Vn„_,, 


o(^-) 


if (a;^ )„-!(,) =0 


\n-\^^_,^ 


o(^<') 


if {Xcr)a-i{t) = 1 


_ fA^"Vm.(c.) 


if Xi 


= 


\N-^q^^(^) 


if Xi 


= 1 


= P{x,x') 







for i = 1,. . . ,N and all a; e S. 

We conclude this section with an application of Lemma |2] that will be useful 
later. 

Corollary 3. Assume that the Markov chain in S with one-step transition 
matrix P given by Iw and M) is ergodic with unique stationary distribution 
TV, and denote by tti 3 its 1,3 two-dimensional marginal. If also pi — P2, then 
7ri,3(0,l)-7ri,3(l,0y. 

Proof. By Lcmmaplwith G being the dihedral group, 7r(xi, X2, X3, X/^, . . . , xn) = 
tt{x3,X2,xi,xn, ■ ■ ■ ,^4) for all cc e E. Now sum over X2 and X4,...,xn to 
obtain the desired result. D 



3 SLLN 

The strong law of large numbers of Ethier and Lee (2009) applies not to the 
Markov chain of Section [2] but to a slightly more informative Markov chain. 
The new state space is E* := E x {1, 2, . . . , N} and the process is in state (a;, i) 
if X describes the status of each player and i is the next player to play. The 
transition matrix P* has the form 



(5) P*{{x,{),{x\j)):= 



Pmi(x) 


if Xi 


= 0, 


^^qm,{x) 


if Xi 


= 1, 



(6) P*((.,.),(.,,)):.r>(^) !;^^ = 0' 

for all {x, i) G S* and j = 1,2, ... ,N, where q„i := 1 — Pm for to = 0, 1, 2, 3, 
and P*{{x,i), {y,j)) = otherwise. 

Lemma 4. Assume that the Markov chain in E with one-step transition matrix 
P given by (Wp and H) is ergodic (see Lemma uj for necessary and sufficient 
conditions) with unique stationary distribution tc. Then there exists S* C E* 
such that, with respect to P* of ISV and Im, S* is closed, irreducible, and 
aperiodic, and all states in S*— 5* are transient. In particular, the Markov chain 
with one-step transition matrix P* is ergodic. Its unique stationary distribution 
■K* is given by 7T*{x,i) := N^^tt{x). 

Proof. By Lemma [l] there exists S* C E such that, with respect to P, S is 
closed, irreducible, and aperiodic, and all states in E — S" are transient. We 
claim that we can take S* := S x {1, 2, . . . , A^}. Given x,y E S and i,j G 
{1, 2, . . . , A^}, we must show that the P*-chain can get from {x,i) to {y,j). 
There are two (possibly overlapping) cases: P* {{x , i) , {x'' , k)) > for all k or 
P*{{x, i), {x, k)) > for all k. (These probabilities do not depend on k and they 
sum to N^^ for fixed fc, so at least one of them must be positive.) In the first 
case, it suffices to note that the P-chain can get from a;* to y. In the second 
case, it suffices to note that the P-chain can get from x to y. This implies the 
stated irreducibility. The other properties follow in a similar way. Since tt is 
stationary for P, we have 

N 



TT*{x,j) = N-\{x) = N-^ Y^ 7r{x')P{x\ x) + N-\{x)P{x, x) 

N N 

= Y.^*{x\t)P*{{x\t),{x,j)) + Y.^*{x,z)P*{{x,z),{x,j)), 



so TT*, defined as in the statement of the lemma, is stationary for P*. D 

Notice also that the profit corresponding to each nonzero entry of P* is 
equal to ±1, so the SLLN holds and there are several formulas for the mean 
parameter, as we now show. 

Theorem 5. Assume that the Markov chain in E with one-step transition ma- 
trix P given by \w and M) is ergodic (see LemmaYufor necessary and sufficient 
conditions) with unique stationary distribution tt. Let {(X{n), I(n))}n>o be 
the Markov chain in E* described above, with an arbitrary initial distribution. 
Define 

U := w{{X{n - 1), /(n - 1)), {X [n) , I {n))) , n>l, 

where the payoff function w is 1 for a win and — 1 for a loss, determined by 
whether the corresponding entry of P* is of the form N~^pm or N~^qm- Let 



Sn := ^1 + • • • + ^n for each n > 1. Then n ^Sn -^ /^^ a.s., where the mean 
parameter jjl^ can he expressed in terms of tv as 



1 ^ 



in terms of certain two-dimensional marginals tti.s = 7r2,4 = • • • = 7rAr_i_i = 
'Kn,2 of TT as 

1 1 

(8) //^ = ^^7ri^3(u,w)(p2u+t, -'?2«+u) 

ti=Ou=0 

= 2[7ri,3(0, 0)po + 7ri^3(0, l)pi + 7ri,3(l, 0)p2 + 7ri,3(l, 1)P3] - 1, 
or in terms of the one- dimensional marginals tti = 7r2 = • • • = tin of tt as 

(9) M'^-7ri(l)-7ri(0)-2^i(l)-l. 

Remark. Another formula for /i^, better suited to numerical computation, was 
given by Ethier and Lee (2012). 

Proof. Theorem 1 of Ethier and Lee (2009), applied to the Markov chain in 
E* with one-step transition matrix P* (restricted to S* of Lemma [4] to ensure 
irreducibility and aperiodicity), gives the SLLN with /i^ = 7v*P*l, where P* 
is P* with each qm replaced by —Qm and 1 := (1,1,..., 1)^, and this implies 
^. Actually, this works directly if the initial distribution is concentrated on 
S* . If it is not, the fact that each state in S* — S* is transient implies that the 
chain reaches S* with probability 1, so the SLLN is unaffected. 

Next, using ([7]) and the rotation invariance property (Lemma pi), we have 



M^ = ]^ XI ^*^^) X[P'"'(==) ~ 9m, (x)] 

xes j=i 

-.Nil 

= -^^^^'^i-l,i+l{u,v){p2u+v ~q2u+v) 
i—1 u—0 v—0 
1 1 

= X X '^^'^{U, V){p2u+v - q2u+v), 



u=Ov=0 



where 7ro,2 '■= '^n,2 and 7r7v-i,Ar+i '■— ■kn-i,1: which implies ([8|. 

Finally, turning to ([9]), we notice that the conditional probability that player 
2 is a winner, given that players 1 and 3 are losers, is not equal to po- Instead we 
have to look back to the last time player 2 played before we condition on the sta- 
tus of player 1 and that of player 3. With this in mind, we let {(X(n), /(n))}„£z 
be a stationary version of the Markov chain in E* with time indexed by Z. Then 

^i(l) = ^2(1) = ^{x : a;2 = 1} = P(X2(0) = 1) 



= J2 PiM-n + 1) = 1, 1{~n) = 2, li-n +1)^2,..., /(-I) ^ 2) 

oo ^ ^ \ n—1 

= E 1 - ]^ P(^2(-n + 1) = 1, /(-n) = 2) 

n=l ^ ^ 

CO , _. s n—1 1 1 

= E 1-]^) EEP(^l(-") = "'^3(-n) = ^;) 

• P(X2(-n + 1) = 1, /(-n) = 2) I Xi(-n) = m, X3(-n) = u) 

ca , _. s n—1 1 1 

= E(^~]^) EE'^l'3("'^)^~^^2u+^ 

n=l ^ ^ M=0 1>=0 

= 7i'i,3(0, 0)po + 7^1,3(0, l)pi + 7ri^3(l, 0)p2 + 7ri,3(l, l)p3, 

where {/(-n) = 2, /(-n + 1) 7^ 2, ... , /(-I) 7^ 2} := {/(-I) = 2} if n = 1. 
Therefore, ^ fohows from ([s]). D 

We conclude with an application of the SLLN. The first conclusion will play 
a minor role in the next section, and the second conclusion is included for 
completeness. 

Corollary 6. Let us refer to the Markov chain in T, with one-step transition 
matrix P given by pp and Q) as the Markov chain with parameter vector 
{P0iPi,P2,P3), and let us denote /i^ of Theorem.]^ by ^'^ {po , pi , p2 , ps) to em- 
phasize its dependence on the parameter vector. 

(a) If the Markov chain with parameter vector {po,Pi,P2tP3) is ergodic, then 
the Markov chain with parameter vector (53, 92; ^ii <Zo) is ergodic, and 

tJ-^{Po,Pi,P2,P3) = -M^ (93, 92,91, 9o)- 

In particular, if po +^3 = 1 and pi + P2 = 1, then /i^(po,Pi,P2,P3) = 0. 

(b) If the Markov chain with parameter vector (po,Pi,P2,P3) is ergodic, then 

2mm{po,pi,p2,P3) - 1 < M^(po,Pi,P2,P3) < 2max(po,pi,p2,P3) - 1- 

Remark, (i) Game A is the special case of game B in which Po — Pi ~ P2 ^ 
P3 = 1/2 and game C is the equally weighted random mixture of game A and 
game B. Let /ig (resp., p^) denote the mean profit per turn to the ensemble 
of iV > 3 players always playing game B (resp., C). Then 

, , Ms — M (P0,Pl,P2,P3), 

Mc :- ^"^((1/2 +Po)/2, (1/2 +pi)/2, (1/2 +p2)/2, (1/2 +P3)/2). 

We say the Parrondo effect is present if ^^ < and fi^ > 0, whereas the 
anti-Parrondo effect is present if ^g > and fj,^ < 0. Part (a) implies that the 
Parrondo effect is present for the parameter vector (po,Pi,P2,P3) if and only if 
the anti-Parrondo effect is present for the parameter vector (93, 92, 91, ^o)- It 



follows that the "Parrondo region" and the "anti-Parrondo region" have the 
same (four-dimensional) volume. 

(a) Part (b) generalizes the obvious identity n^ {p,p,p,p) = 2p — 1. 

Proof, (a) This was proved by Ethier and Lee (2012) using a coupling argument. 

(b) Here we use another coupling. Let f/i, C/27 • • ■ be i.i.d. uniform (0, 1) ran- 
dom variables, and let /i, J2,... be an independent sequence of i.i.d. uniform 
{1,2,..., N} random variables. Then our Markov chain {X{n)} can be con- 
structed in such a way that, for each n > 1, X(n) is a function of X{n — 1), /„, 
and Un with 

n 

fc=l 

representing the total profit after n turns to the ensemble of N players. With 
S^ := ELi[2 • Uo.p]iUk) - 1], we have 

^min(po,Pl,P2,P3) < 5* < 5'max(po,pi,P2,P3) 71 > 1 

so the desired result follows from Theorem [5] and the i.i.d. SLLN. D 

4 The case pQ = 1, p^ = 

Suppose pq ~ 1 and p^ ~ 0. If we also assume that Pi = P2 G (1/2, 1), then we 
can confirm the Parrondo effect for all even A^ > 4. 

Theorem 7. Let po — I, Pi — P2 ^ (1/2, 1); and ps — 0. Let fi^ (resp., ^^ ) 
denote the mean profit p er t urn to the ensemble of N > 3 players always playing 
game B (resp., C); cf. (10). Then fi^ — for all even N > 4, /x^ > for all 



odd N > 3, and ji^ > for all N > 3. In particular, the Parrondo effect is 
present if and only if N is even. 

Proof. First, we compute /i^ assuming only pq = 1, pi,P2 G [0, 1], ps = 0, and 
< pi + p2 < 2. 

If N is even, then the two states 01 • • • 01 and 10 • • • 10, in which Os and Is 
alternate, are absorbing, and all other states are transient (see LemmafTl). From 
either of these two states there is a win of one unit if the player chosen to play 
is a winner (probability 1/2) and a loss of one unit if the player chosen to play 
is a loser (probability 1/2). Consequently, the i.i.d. SLLN applies and fig — 0, 
regardless of pi and p2 . 

On the other hand, if N is odd, then the set of 2N states in which Os and Is 
alternate, with the single exception of a pair of adjacent Os or a pair of adjacent 
Is, is closed, irreducible, and aperiodic, and all other states are transient (see 
LemmafTl). Let us order the states in this set as follows: First, the states with 
adjacent Os are ordered by the position of the first of the adjacent pair in 
clockwise order (e.g., 001 • • • 01 is first, 010 • • • 10 is A^th). Next, the states with 
adjacent Is are ordered by the position of the first 1 of the adjacent pair in 
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clockwise order. With this ordering, the 2N x 27V one-step transition matrix 
obtained by restricting P to this set has the block form 

N\ P«-«i {N-q^-q2)lN 

where P^^ '^^ is the N x N matrix with each entry of the superdiagonal as well 
as the {N, 1) entry equal to pi, and each entry of the subdiagonal as well as the 
(l,iV) entry equal to p2- By Lenima[2| the unique stationary distribution tt for 
this Markov chain has the form 

TT — {aN,aN, ■ ■ ■ ,aN,PN,PN, ■ ■ ■ ,Pn), 

so TT = TvP and ttI = 1 result in ajy = {qi+q2)/{2N) and (3]\[ — {pi +p2)/{2N). 
Finally, by ^, 

P1+P2-I 



Hb =2^7f(2fc)-l = 2 



A^-l A^ + 1 ^ 

7. OIN H 7. PN 



- 1 = 



N 



fc=i 

In particular, this proves the assertions about /x^ when Pi — P2 ^ (1/2, !)• 
It remains to show that /ip > under this assumption, regardless of the parity 
of iV. 

Now n^ = Ai^(3/4, (1/2 +pi)/2, (1/2 +pi)/2, 1/4), which by Corollary [e] is 
at pi = 1/2. If we could show that this function is increasing in pi, the proof 
would be complete. However, despite being very plausible, this monotonicity 
property appears difficult to prove. We can prove it computationally for small 
N. For example, 

r(2pi-l)/5 if AT = 3, 

Mc = I 7(2pi - l)/(53 - 16pi + 16p?) if A^ = 4, 

[{2pi - 1)(65 - 14pi + 14p2)/(423 - 134pi + 134^^) if Ar = 5. 

But a noncomputational proof is needed. Fortunately, there is an alternative 
approach that avoids the monotonicity question. 

Let 7ri^3 be the 1,3 two-dimensional marginal of tt when the probability 
parameters are 3/4, {l/2+pi)/2, (1/2 +pi)/2, and 1/4. We apply Theorem|5] 
twice. By ^ and CoroUary[3J 

(11) ^l^ = ^1,3(1, 0) + ^1^3(1, 1) - [^1^3(0, 0) + 7ri^3(0, 1)] 

= ^1,3(1,1) -^1,3(0,0). 

By (|8|, Corollary [Sj and ([IT]), 



/x^ = ^1,3(0, 0)(l/2) + ^1,3(0, l)(2pi - 1) + 7ri,3(l, l)(-l/2) 
= (2pi- 1)^1,3(0, l)-(l/2)/i^. 

Therefore, fj,^ — (2/3)(2pi — l)7ri^3(0, 1), and this is positive by the irreducibility 
of the Markov chain with the stated probability parameters and the assumption 
that pi > 1/2. n 
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Under the assumptions of Theorem JTJ we know that n ^Sn ^ ^^ a.s. (by 



the i.i.d. SLLN in the case of game B and even N^ by Theorem [5] otherwise), 
hence n~^E[S'„] — > ^^ . But the rate of convergence here depends on TV and 
of course on the initial distribution. This is illustrated in Figure [T] in which 
A^ = 12 and 13, Pq = I, Pi = P2 = 3/4, and ps = 0. We assume an initial 
distribution that is uniform on S and use So '■= and 



(12) 



N-^^Sr 



nN 



■N 



1 



N ^^ 



tvqP"'-^P1, 



1,2, 



100, 



where ttq is the 2^-dimensional row vector with every entry equal to 2^''^, P is 
the 2^ X 2^ one-step transition matrix from (llj) and ([2|, P is P with each g^ 
replaced by —(?,„, and 1 is the 2^-dimensional column vector of Is. We evaluate 



( 12 ) by recursion, not by matrix arithmetic, because the matrices are rather 



large. 




30 40 50 60 70 

(number of games)/N 



Figure 1: Illustration of Theorem [7] in the special cases iV = 12 and 13 and 
Po = 1, pi = P2 = 3/4, and ps = 0. The initial distribution is assumed 
uniform on E. The Parrondo effect is present for A^ = 12 because fi^ = and 
li}? R:! 0.0766019. It is not present for A^ = 13 because fi]^ = 1/26 w 0.0384615 
and ^}? « 0.0766021. (The open squares are hidden behind the open circles.) 



12 



5 SLLN in continuous time 

The Parrondo effect is said to occur if there is a reversal in direction in some sys- 
tem parameter when two similar dynamics are combined. The term "combined" 
is fairly unambiguous when time is discrete but less so when time is continuous. 
The original examples of the paradox all had the following form. Game A is fair 
or losing, game B is fair or losing, but the combination of the two games, game 
C, is winning. Here game C has two possible interpretations. It could be the 
random mixture of game A and game B, denoted by C := jA + (1 — j)B, where 
< 7 < 1; with probability 7, game A is played, and with probability 1 — 7, 
game B is played. Or it could be a nonrandom pattern such as C :— A''B'^; 
game A is played r times, then game B is played s times. In these examples, 
games A and B are controlled by Markov chains in a state space Sq with one- 
step transition matrices Pa and Pb- Then game C is controlled by Markov 
chains in Sq with one-step transition matrices 



(13) Pc:=jPa + {1-7)Pb and Pc :^ P^P, 



B 



in the random-mixture and nonrandom-pattern cases, respectively. 

Now suppose that games A and B are controlled by continuous-time Markov 
processes in a compact state space E with Feller semigroups {TA(i)} and {TB{t)} 
on C{E) having generators ^a and ^s, at least on a domain D C C{E). The 



analogues of ( 13 1 are 

(14) [lTA{lln) + (1 - 7)TB(l/n)]L»*J/ ^ Tc(t)f 
and, with 7 := r/{r -f s), 

(15) [TA{lln)TB{{l - 7)/n)]L"*J/ ^ Tc(<)/, 

where {T'c(t)} is the Feller semigroup whose generator is assumed to be the 



closure of J^c '■— 1-^A + (1 — i)-^b acting on D. The hmit (14 1 is justified by 



Chernoff's product formula, and the limit (151 is justified by Trotter's product 
formula. So, in continuous time, there is no distinction between the random- 
mixture and nonrandom-pattern cases, at least in the limit as the time allotted 
to each game tends to 0. See Montero (2011) for a different approach to the 
same issue. 

Let us consider whether the SLLN applies to the ensemble's profits in the 
continuous-time analogue of the A^-player model in the state space S or, better 
yet, S* := S X {1, 2, . . . , TV}. The discrete-time Markov chain in E* has one-step 
transition matrix P* given by ^ and ^. We assume that the conditions for 
ergodicity in Lemma [I] are satisfied. 

The continuous-time analogue of this process has infinitesimal matrix Q* := 
P* — J, and it can be constructed as follows: Let {X*(n)} denote the discrete- 
time Markov chain in S* and let {N{ty\ be an independent, rate 1, Poisson 
process. Then Y*{t) ;= X*{N{t)) defines a continuous-time Markov chain 
{Y'*(i)} in S* with infinitesimal matrix Q* . 
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There is a technical issue with continuous-time Markov chains, namely that 
jumps from a state to itself are not recognized as jumps. In our original model 
with state space S, when a winner wins or a loser loses, there is a jump from 
the current state to itself in the discrete-time model and no change of state in 
the continuous-time model. By augmenting the state space to E*, this problem 
is minimized, but when a winner wins and plays again at the next turn or when 
a loser loses and plays again at the next turn, we have a jump from a state (a;, i) 
to itself in the discrete-time model and no change of state in the continuous- 
time model. These "fictitious" jumps are needed to define the cumulative-profit 
process, in terms of which the Parrondo effect is defined. By representing the 
continuous-time Markov chain in terms of a Poisson process as above, these 
fictitious jumps become recognizable. 

To be more explicit, the above representation allows us to think of the 
continuous-time Markov chain as having jumps from state {x,i) at exponential 
rate 1 as follows: If x.i = 0, the process jumps to state {x^,j) with probability 
N~^Pmi{x) and to state (x^j) with probability N~^q^.(^x)j if ^i = li it jumps 
to state {x^,j) with probability A^~^9mi(a;) and to state {x,j) with probabihty 
N~^Pmi(x)- Fictitious jumps from state {x,i) to itself occur with probability 
N~^qmi(x) if ^i = and with probability N~^pmi(x) if ^i = 1- 

Let Sn be the cumulative profit to the ensemble of N players after n steps 
of the discrete-time model. Theorem [s] tells us that rr'^Sn — > /^^ a.s. Then 
Si^u) is the cumulative profit at time t to the ensemble of N players in the 
continuous-time model, and we have 

^N(t) _ Snu) N{t) ^ 
t N{t) t ^ 

as i — >■ CXI, where /i^ is the mean profit per turn in the discrete-time model. Let 
us summarize this result as follows. 

Theorem 8. Assume that the conditions for ergodicity in Lemma^are satisfied. 
Let St be the cumulative profit at time t to the ensemble of N players, assuming 
the continuous-time Markov chain model in S* with infinitesimal matrix Q* := 
P* — I, where P* is the one-step transition matrix given by Oj and M). Then 
lim(_j.oo i^^Sf = p,^ a.s., where fi^ is as in TheoremlA 

Implicitly, we have been discussing game B. Game A is the special case 
Pq = Pi = P2 = Pa — 1/2. By the discussion at the beginning of this section 
with 7 = 1/2, game C is simply game B with pm replaced by (1/2 -I- Pm)/'^ for 
771 = 0, 1, 2, 3. Thus, the Parrondo effect appears in continuous time if and only 
if it appears in discrete time. 

6 A spin system 

We want to show that our discrete-time Markov chain converges in distribution, 
after rescaling its time parameter, to an interacting particle system, or more 
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specifically a spin system on the one-dimensional integer lattice Z. Let us begin 
by characterizing the limiting process in terms of its generator. Its state space 
is the product space 

{0, 1}^ := {x = {. . . ,x^2,X-i,xq,xi,X2, . . .) : Xi <E {0, 1} for all i G Z}. 

We will usually refer to Xi as the status (loser or winner, or 1) of player 
i; occasionally, it will be convenient to refer to it as the spin at site i. Let 
mi{x) := 2xi-i +Xi^i as before but without the periodic boundary conditions. 
Also, let x^ be the element of {0, 1}^ equal to x except at the zth component. 
For example, a;" :=(... , x_2, x^i, 1 — xq, xi, X2, ■ ■ .). 

The generator depends on the four probability parameters Pq,Pi,P2tP3 G 
[0, 1], and it has the form 

ieZ:Xi=0 ieZ:Xi=l 



E 



c,{x)[f{x^)-f{x)] 



for functions / depending on only finitely many components, where the flip rates 
are given by 

Mc\ / ^ J Prni(x) n Xi U, 

(16) c^{x) := <^ -r 1 

and Qm ■= 1 — Pm for m ~ 0,1,2,3. This is of the form (IIL0.5) of Liggett 
(1985). The sufficient condition (III. 0.3) of Liggett for the characterization of 
the process is, in our notation, 

sup 2^ sup \Ci{x) — Ci{X'' )\ < oo , 

which is trivially satisfied because the summands are unless |i — j| < 1, hence 
the expression on the left is at most 3. It can be shown that the functions 
depending on only finitely many components form a core for the generator of 
the Feller semigroup associated with the process. 

Next we would like to justify the claim that this spin system is the limit in 
distribution of the A^-player model after an appropriate time change. First, it 
is convenient to relabel the N players. Instead of labeling them from 1 to N, 
we label them from -(TV - l)/2 to {N - l)/2 if N is odd, and from -7V/2 to 
N/2 — 1 if iV is even. In general, we label the players from Ij^ to rjy, where 

i-(N -l)/2 if iV is odd, , i(N -l)/2 if iV is odd. 

In ■= { , and r^ := < 

\-N/2 if N is even, [N/2 - 1 if iV is even, 

with the understanding that players Ipf and rj\j are nearest neighbors. The state 
space is 

Sat := {x = {xij^,.. . ,x_i, a:o,a;i, . . . , x^.^ ) : a;^ G {0,1} for i = /at, . . . , tat}. 
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(This is what we previously cahed S but with the players relabeled. To avoid 
confusion, we make the dependence on N explicit in the notation.) We also 
speed up time in the iV-player model so that A'' one-step transitions occur per 
unit of time. The resulting discrete generator has the form 

{^Nf){x) := NE[f{XN{l)) - fix) I Xn{Q) = x] 






N 

■ lN<.i<.rN:Xi—0 



lj\[<i<rjs[:xi^l 

where Xi^-i :— Xrj^ and x^^+i '.— xi^. Consequently, if we define r]js[ : 
B({0, 1}Z) K^ i?(S^) by 

{VNf){xi,^ ,...,Xr^):^f{---,0,0,Xij^,...,Xrj,,0,0,...), 

then {^NVNf){x) = r]N{^f){x) for all x e J^n and N > 2K + 4, where f{x) 
depends on x only through the 2K + 1 components Xi, —K < i < K. 

This implies that the process {Xis[{lNt\)} converges in distribution to the 
spin system {X{t)} (e.g., Theorems 1.6.5 and 4.2.6 of Ethier and Kurtz 1986). 
More importantly, it shows that, if the spin system has a unique stationary 
distribution, then the unique stationary distribution of the iV-player Markov 
chain (assumed ergodic in the sense of Lemma[I]) , converges to it in the topology 
of weak convergence (essentially Proposition 1.2.14 of Liggett 1985). Let us 
assume that the spin system has a unique stationary distribution tt, and let us 
denote the unique stationary distribution of the A^-player Markov chain by tt^ . 
(We previously denoted the latter by tt but now it is necessary to make the 
dependence on N explicit. We do not use boldface for n^ or tt because it is 
no longer useful or possible, respectively, to think of them as row vectors.) Let 
us denote their —1,1 two-dimensional marginals by tt^-^ -^ and tt-i^i. Then we 
have 

(17) ^^i.i(0,0)po + ^^14 (0, l)Pi + ^^1,1 (1, 0)P2 + ^^1,1 (1, 1)P3 

-^ 7r_i,i(0, 0)po + 7r_i,i(0, l)pi + 7r_i4(l, 0)p2 + 7r-i4(l, l)p3, 

hence fi^ , the mean parameter in Theorem pi converges as iV — > oo to a limit 
that can be expressed in terms of the spin system. It remains to show that this 
limit can be interpreted as a mean profit for the spin system; we return to this 
point in Section [7j 

Under what conditions does the spin system have a unique stationary dis- 
tribution (also referred to as a unique invariant probability measure)? We will 
give sufficient conditions for the spin system to be ergodic, which means not only 
that there is a unique stationary distribution tt but that the process at time t 
converges in distribution to tt as t — >■ oo, regardless of the initial distribution. 
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Theorem 9. With pQ,pi,p2,P3 € [0,1]; the spin system on Z with flip rates 
pm is ergodic if at least one of the following four conditions is satisfied: 

(a) (basic estimate applies) 

(18) max(|po- Pi 1,1^2 -p3|)+max(|po -p2\,\Pi - Psl) < 1; 

(b) (attractiveness or repulsiveness applies) 

(19) < min(po,P3) < min(pi,p2) < niax(pi,p2) < max(po,P3) < 1; 

(c) (coalescing duality applies) 

(20) niax(pi,p2,P3,Pi +P2-P3)-P3 < Po/2 < min(pi,p2,P3,Pi +P2-P3); 

(d) (annihilating duality applies) 

(21) po,Pi,P2,P3e (2p-l,2p)n(0,l), p:=(po+Pi+P2+P3)/4. 

Remark, [i) Absorbing states are compatible with ergodicity as long as there is 
only one of them. In cases of two or more absorbing states, none of the conditions 
(a), (b), (c), or (d) is satisfied. There are six such cases: If (poiPs) = (0, 1), then 
and 1 are absorbing; if (po,P3) — (1, 0), then the two states with alternating Os 
and Is are absorbing; if (po,Pi,P2) = (1, 0, 0), then the three states of the form 
(• • • 001001001 • • • ) are absorbing; if (pi,_P2,P3) = (1, 1, 0), then the three states 
of the form (• • • 011011011 • • • ) are absorbing; if (po,Pi,P2,P3) = (1,0,0,0), then 
states with no adjacent Is and no more than two consecutive Os are absorbing; 
if (po,Pi,P2,P3) = (1, 1, 1, 0), then states with no adjacent Os and no more than 
two consecutive Is are absorbing. (In the last two cases, the set of absorbing 
states is uncountable.) 

[ii) None of the four conditions on (po,Pi,P2,P3) is implied by the other 
three. For example, (1/4, 1/4, 1/4,0) satisfies (a) only, (1/4, 1/4, 1/4,3/4) satis- 
fies (b) only, (1/4,3/4,3/4,1) satisfies (c) only, and (1/4,3/4,3/4,1/4) satisfies 
(d) only 

{Hi) Under the assumption that pi = P2, the (three-dimensional) volumes of 
the regions described by (a), (b), (c), and (d) are respectively 7/12, 1/3, 7/32, 
and 2/3. The volume of the union of the four regions is 3323/4032, representing 
about 82.4% of the parameter space. (See the remark following the statement of 



Corollary 14 for a slight improvement.) Without the assumption that pi = P2, 
the (four-dimensional) volumes of the regions described by (a), (b), (c), and 
(d) are respectively 7/12, 1/6, 65/384, and 2/3. The volume of the union of 
the four regions could not be evaluated, but can be estimated via simulation to 
represent about 78.9% of the parameter space. 

Proof, (a) uses condition (III. 0.6) of Liggett (1985). In our notation, that con- 
dition is 

sup y^ sup \ci{x) ~ Ci{x' )\ < inf [^(a;) -f Cj(a;')]. 
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The right side of this inequality is inino<m<3(pm + <im) = 1- So the incquahty 
is equivalent to (a). 

(b) uses a result of Gray (1982): A translation-invariant nearest-neighbor 
spin system on Z is ergodic if it is attractive or repulsive and if it has (strictly) 
positive flip rates. The nearest-neighbor property requires that, for each i € Z, 
the flip rate at site i depend only on the spins at sites i — 1, i, and i + I. 
Translation invariance (deflned precisely below) strengthens this by requiring 
that, for each z € Z, the flip rate at site i depend only on the spins at sites 
i — 1, i, and i + 1 in a way that does not depend on i. Attractiveness (resp., 
repulsiveness) is the requirement that whenever x < y componentwise and 
i G Z, 

Ci{x) < (resp., >) Ci{y) if Xi = y^ = 0, 
Ci{x) > (resp., <) Ci{y) if Xi = y^ = 1. 

In our case, attractiveness reduces to po < inm{pi,p2) < max(pi,p2) < Ps, and 
repulsiveness reduces to ps < min{pi,p2) < max(pi,p2) < Poj hence (b) suffices, 
since the outer inequalities ensure positive flip rates. 

(c) and (d) use coalescing and annihilating duality, respectively, as described 
in Section III. 4 of Liggett (1985). Since our flip rates are translation invariant 
and nearest neighbor, there are nine parameters necessary to specify (III. 4. 3) 
and (III. 4. 4) of Liggett, namely (using his notation) c(0) = c{x) and p(0. A) = 
p{x,x + A) as A ranges over the eight subsets of {—1,0, 1} and furthermore 
each such A may be augmented by including oo. The basic requirement of 
our spin system is that Ci{x) + Ci{x^) = 1 for all x G {0,1}'^ and i G Z, 
and this implies that three of the eight probabilities are (namely, the ones 
corresponding to A n Z = {-1, 0}, yl n Z == {0, 1}, and A n Z = {-1, 0, 1}) 
and c(0) is determined. This leaves five remaining parameters, which we will 
denote by z^, z_i, zq, zi, and z_i.i, the interpretation being that za =p{Q,A) 
if -za > and za — —p{Q, A U {oo}) if za < 0. They must satisfy 

(1 - zo)"^[l - (z0 + z_i + zo + zi + z_i,i)] = Po 
{1- zoy^[l- iz0+z^i+za)]=pi 

(1 - Zo)"^[l - {Z0 + Zo +Zl)] =P2 

{l-zoy^[l-{z0 + zo)]=P3 

in the case of coalescing duality, and 

2"^(1 - zo)"^(l + Z0 - z_i - Zo - zi + z^ia) = Po 
2"^(1 - zo)"^(l + Z0 - z;_i - Zo + zi - z-ij) = pi 

V / 1 1 

2" (1 - Zo)" (1 + Z0 + Z_l - Zo - Zl - Z^ia) = P2 

2"^(1 - zo^'^il + Z0 + z_i - Zo + Zl + z-ia) = pz 
in the case of annihilating duality. In either case, if there is a solution with 
(24) |z0| + |z_i| + |zo| + |zi| + |z_i,i| < 1, 
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then Theorem III. 5.1 of Liggett (1985) ensures the ergodicity of the spin system. 
The hncar systems are underdetermined, and a solution to (22) is given by 



z-i = 



{P3 -Pi)z0 



Z\ 



(P3 



-Vi 

V2)Zq5 



1 -P3 



Zo = 1 - 



Z-\.\ 



Z0 

l-Ps' 

bl + P2 



{po+P-i)]z0 



1-P3 



provided pa < 1. We must have Z0 > to ensure \zq\ < 1. We can also require, 
without loss of generality, that zgi < 1 — p^, so that zq > 0. Then z_i, zi, and 
Z-i^i can have any combination of signs, so to evaluate the sum in (24 1 we must 



consider eight cases. Similarly, a solution to (23) is given by 



Z-l 



Zl 



[P2+P3 - {PQ+Pi)]z0 



Pa+Pi+P2+P3-^ 

[Pl +P3- {PO+P2)]Z0 



zo 



2zo 



Pa+Pi+P2+P3-'2 



-1,1 



Po 
[Po- 



-Pl 

P3- 



-P2 - 
(Pl 



P3-2' 

-P2)]Z0 



PO+P1+P2+ P3 



provided Po + Pi + P2 + P3 7^ 2. We must give Z0 the same sign as Po + Pi + 
P2 +P3 ^ 2 to ensure |zo| < 1. We can also require, without loss of generality, 
that Z0/{po + Pl + P2 + P3 ~ 2) < 1/2, so that zq > 0. Then Z0, z_i, zi, and 
Z-i^i can have any combination of signs, so to evaluate the sum in (24 1 we must 
consider 16 cases. 

We can now state the results. In the case of coalescing duality, let 

C-i ■■^{p3>Pi}, Ci := {p3 > P2}, C_ia := {pi +P2 >Po+P3}, 

and define CZi, C^ , and CZi 1 to be the same sets with inequalities reversed 
(> becomes <). Then a sufficient condition for ergodicity is that {potPitP2tP3) 
belong to 

(25) C:=[c_inCinc_i,in{po>o}] 

u [c_i n Ci n czi^^ n {pi +P2> pa/2 + ps}] 

u [c_i n cr n c_i,i n {po/2+p3 > P2}] 

u [c_i n cr n czi^^ n {pi > po/2}] 

u [czi n Ci n c_i4 n {po/2 + pa > pi}] 

u [czi n Ci n czi^-^ n fe > po/2}] 

u [czi n cz n c_i,i n {po/2 + 2^3 > pi + P2}] 

u [CZi n cz n cz^^i n {pa > po/2}]. 

From this it is straightforward to show that C is the intersection of the eight 



sets that appear in braces in ( 25 ) . It follows that C can be expressed as in ( 20 ) 



In the case of annihilating duality, let 

D0 ■■= {po +Pi +P2 +P3 > 2}, D_i := {p2 +pa > Po +Pi}, 
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Di := {pi +P3>po+ P2}, i^-1,1 := {pa + P3 > Pi + P2}, 



and define D^, DZi, D^ , and DZi i to be the same sets with inequahties 
reversed (> and > become < and <, respectively). Then a sufficient condition 
for ergodicity is that (po,Pi,P2,P3) belong to 

(26) D := [D0 n £>_! n £>! n D-i,! n fe < i}] 

u [D^ n D-i n £>! n DZi ^ n {2 + po > pi + P2 + ps}] 

u [£'0 n D-i DD^n L»_i,i n {2 +pi > po +P2 +P3}] 

u [D0 n D-i n Dj^ n Di^i n {p2 < 1}] 

u [D0 n z?Zi n i?i n i?_i4 n {2 + ^2 > po + pi + Pa}] 

u [i:>0 n DZi n Di n 151;^;^ n {pi < 1}] 

u [D0 n DZi nDz n d_i,i n {po < 1}] 

u [D0 n DZi nDzn dz^^^ n {2 +^3 > po +pi +P2}] 

u [^0 n D^i n Di n D^i^i n {po + Pi + P2 > p^}] 

u [D0 n £>_! n z?! n DZi ^ n {po > 0}] 

u [D0 n £>_! n Dj^ n d_i,i n {pi > 0}] 

u [D0 n £>_! n Dj^ n Din n {po +pi +P3 > P2}] 

u [D0 n DZi n Di n i:)_i,i n {p2 > 0}] 

u [D0 n LTi n Di n DZi ^ n {po +P2 +P3 > pi}] 

u [1)0 n DZi r\D^r\ D^i,i n {pi +P2 +p3 > po}] 

u [£'0 n DZi n Df n l>Zi,i n {pa > 0}]. 

From this it is reasonably straightforward to show that D is the intersection of 



the 16 sets that appear in braces in (26 1. It follows that D can be expressed as 



in (21). (The use of DZi instead of DZi, for example, allows using symmetry 
in the derivation.) 

Recall that we assumed pa < 1 for coalescing duality and po +pi +P2 +P3 7^ 2 
for annihilating duality. Neither assumption is necessary, as we now show. 



If P3 = 1, then we can solve (22) to get 

Z0 = 0, z_i = (1 -pi)(l - zq), 
zi = (1 -P2)(l - zo), z^i.i = -(1 +po -Pi -P2)(l - za). 

P2I < 1, which 



The inequality \2A\ holds if |1 
re( 

P3 



Pil 



|1 -P2I + |1 +po -pi 



requires < po < 2(pi +p2 — 1). But this is the same requirement as (20 1 with 
= 1. 
If P3 ::= 2 — (po +Pi +P2), we can solve (23) to get 



Z0 = 0, 



Zq = 1 



Z\ = 



{1-P0~P2)Z. 

1 - Po - Pi 



1 - Po - Pi 



2-1,1 



(1 -pi -P2)Z-1 

1 - Po - Pi 
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provided po+Pi ^ 1- z-i must have the same sign as 1—po—pi to ensure that 
|2;o| < 1. We can also require, without loss of generality, that Z-i/{l—po—pi) < 
1, so that zq > 0. Then z_i, zi, and 2;_i,i can have any combination of signs, 
so to evaluate the sum in p4| ) we must consider eight cases. Let 

£>-! :={Po+Pi < 1}, Di:={po+P2<l}, D^i^i := {pi + p2 < 1}, 

and define DZi, D^ , and DZi i to be the same sets with inequalities reversed. 
Then a sufficient condition for ergodicity is that {po,pi,p2,P3) belong to 



(27) 



D 



[i?_i n i?i n D_i,i n {po + Pi + P2 > i}] 
u [£>_! n Di n DZi^i n {pa > o}] 

u [D^i nDz n D-i,! n {pi > o}] 
u[D^inDznDZi^,n{p2<i}] 
u [DZi n Di n D^i^i n fe > o}] 

U [DZi nDiH DZi^i n {pi < 1}] 

u [DZi nDzn D-i,! n {po < i}] 
[j[DZinDzr\DZi^in{pQ+pi+p2 <2}]. 

From this it is straightforward to show that D is the intersection of the eight 



sets that appear in braces in ( 27 ) . The result is that ergodicity holds if pq + 
Pi + P2 + P3 = 2, Pq + Pi 7^ 1, and Po,Pi,P2,P3 € (0, !)• This is consistent with 



(21) 



Finally, we must remove the restriction that po+pi j^ 1. So we consider the 
case in which po + Pi — ^ and P2 + Pa = 1- If Pi := 1 ^ Po and p2 :— 1 — pa, we 



can solve ( 23 ) to get 



Z0 



z_i =: 0, zi = (p3 -po)(l-zo). 



(P0+P3 - 1)(1 -^o)- 



With < zo < 1, (24) holds because |p3 — Po| + |po + Ps — 1| < 1, assuming 
only that < po < 1 and < P3 < 1. The result is that ergodicity holds if 
Po + Pi = 1, P2 +P3 = 1, and Po,Pi,P2,P3 G (0, 1). This is also consistent with 

m. D 



Remark. The strict inequalities in (25) and (26) are the requirements for (24 1 
to hold in the various (8 or 16) cases. If one of these conditions fails because 
equality holds instead of strict inequality, then ergodicity is assured if in addition 
\z0\ > |z_i_i|, by virtue of Corollary III. 5. 8 of Liggett (1985). For example, 
consider Toral's (2001) case, (po,Pi,P2,P3) = (1,4/25,4/25,7/10). This belongs 
to D0 n DZi n Dj" n D-i,! (line 7 of ((26|) but po = 1. Here z_i4 = 69z0, so 



the additional condition is not met, and annihilating duality is inconclusive in 
this case. 

Again we have described game B above. Game A is the special case po = 
Pi = P2 = P3 = 1/2, and game C, by virtue of the discussion at the beginning 
of Sectionlsj is game B with pm replaced by (1/2 +p,„)/2 for ttt, = 0, 1, 2, 3. We 
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note that condition (a) often fails for game B but nearly always holds for game 
C because, if pm is replaced by (1/2 + Pm)/'^ in condition (a) for m = 0, 1, 2, 3, 
the requirement becomes 

(28) max(|po- Pi 1,1^2 -pal) + max(|po -P2\,\Pi -Psl) < 2, 

which nearly always holds. This result is worth emphasizing in the form of a 
theorem. 

Theorem 10. (a) Assume that {pQ,pi,p2,P3) is such that Theorem]^ applies 
to define fig :— /i^(po,Pi,P2,P3) for all N > 3, and that the spin system on Z 
with flip rates (16) is ergodic (see Theoreml^for sufficient conditions). Then 
limAr_j.oo Ms exists. 

(b) Assume only that {po,Pi,P2,P3) is not equal to (0,0,1,0), (0,1,0,0), 
(1,0,1,1), or (1,1,0,1). Let fi^ := ^"^((1/2 + po)/2, (1/2 + pi)/2, (1/2 + 
P2)/2, (1/2 + P3)/2), where fj,^ is as in TheoremlA T/ien linijv-foo /^c ^^i^ts. 

Remark. In the proof of part (b), we could use condition (d) of Theorem^ in 
place of condition (a) of that theorem. The result is the same. 

Proof. The only step that remains to be checked is that, under the assumptions 
in part (b), ergodicity holds when the parameter vector is ((1/2 +po)/2, (1/2 + 



Pi)/2, (l/2+p2)/2, (l/2+p3)/2). Condition ^ will fail if and only if (po,Pi,P2) 
equals (0,1,1) or (1,0,0), (po,Pi,P3) or {po,P2,P3) equals (0,1,0) or (1,0,1), 
or {pi,p2,P3) equals (0,0,1) or (1,1,0). In the first case, {po,Pi,P2) = (0,1,1), 
condition (b) of Theorem [9] applies if p3 = 1, and condition (d) applies if < 
P3 < 1. The second case and the last two cases are treated similarly. As for 
{po,Pi,P3) = (0,1,0), p2 = is ruled out by assumption, and condition (d) 
applies if < P2 < 1- The last three cases are treated similarly. D 

We conclude this section by showing that the invariance under rotation (and, 
if Pi = p2, reflection) that we found in the A^-player model (see Lemma ^ and 
the discussion following it) carries over to the spin system, except that rotation 
invariance becomes translation invariance. We could prove this as a consequence 
of the weak convergence of tt^ to n, but instead we give a direct proof. 

Define r : Z i-^ Z by T{i) ■.= i + l and for x € {0, 1}^ define x^ e {0, 1}^ by 

Ct(z)- 



(Xr)i := XtU). Finally, define r : {0,1}^ H- {0,1}^ by t{x) :— Xt-. Then (cf. 



\%) Ci{x) = c^-i(i)(T(a;)) for all x G {0, 1} and i G Z, and therefore 
(29) i^(/or) = (i^/)or 



for all / depending on only finitely many components. (This can be taken as the 
definition of translation invariance.) Similarly, define p : Z t-^ Z by p{i) := —i 
and for x e {0,1}^ define Xp e {0,1}'^ by {xp)i :— Xp(^iy Finally, define 
p : {0,1}^ 1-^ {0,1}^ by p{x) := Xp. If pi = p2, then ^(a;) = Cp-i(j)(p(a;)) for 
all X G {0, 1}^ and i G Z, and therefore 



(30) J^if o p) = (if/) o p 
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for all / depending on only finitely many components. (We might call this 
reflection invariance.) 



Lemma 11. // the spin system on Z with flip rates [16) is ergodic with unique 
stationary distribution it, then tt = ttot"^. Under the same assumption, if also 
Pi — P2, then TT — TT o p^^ . 



Proof. From ( 29 ) , we have 

0- f^{f0T)dTT^ f{^f)0TdTT^ / ^/ d(7r O T^l) 

for all / depending on only finitely many components, so tt o r^^ is stationary. 



The second conclusion follows from ( 30 1 in the same way. D 



Corollary 12. If the spin system on Z with flip rates (16) is ergodic with unique 
stationary distribution it, and if also pi = P2, then the —1,1 two-dimensional 
marginal TT^ I ^i of tt satisfies tt^ii{0,1) = 7r_i^i(l, 0). 



Proof. This is immediate from the second conclusion of Lemma 11 D 



7 SLLN for the spin system 

The approach used in Section [5] does not apply here. One problem is that the 
cumulative profit to the (infinite) ensemble of players is ill-defined (being an 
infinite sum of ±1 terms) and changes instantaneously. So instead we consider 
the cumulative profit to a single player (or to a finite set of players). This, of 
course, is non-Markovian. 



Theorem 13. Assume that the spin system on Z with flip rates (16) is ergodic 
with unique stationary distribution tt. Let St be the cumulative profit at time t 
to player who, along with infinitely many other players, plays according to the 
spin system model. Then limf_j.oo t~'^St = /^ a.s., where the mean parameter jjl 
can be expressed in terms of certain two-dimensional marginals ■ ■ ■ = 7r_2.o = 
7r-i,i = 7ro,2 = •• • ofTTas 

1 1 

(31) H == y^ y^ TT^i^i{u, V){p2u+v - q2u+v) 

u=0 -0=0 

= 2[^_i,i(0,0)po + vr_i,i(0,l)pi 

+ 7r_i,i(l,0)p2 + 7r_i,i(l, 1)P3] - 1, 

or in terms of the one- dimensional marginals ■ ■ ■ — n^i — ttq — tti — ■ ■ ■ of it 
as 

(32) M = 7ro(l)-^o(0) = 2^o(l)-l. 
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Remark, {i) There is nothing special about player 0, so he can be replaced by 
any player i £ Z. In fact, for arbitrary finite Ac Z, let Sf be the cumulative 
profit per player at time t to the ensemble of players belonging to A. Then 
limt_>oo t~^^t ~ M ^-S-, as a consequence of the theorem. 



(ii) Notice that /z coincides with hmjv-i-oo fJ-^ by (171 and Theorem [s] so we 
now know what the limits in Theorem 1101 are. 

Proof. Let {X{t)} be a stationary version of the spin system, that is, X(0) has 
distribution tt. Then, by the ergodic theorem for stationary processes. 



lim - / f{X{s))ds^ / f{x)'K{dx) a.s. 

-*°° t Jo J{o.n^ 



"'{0,1} 

for all / G i^(7r). In particular, with / being the indicator of {x e {0, 1}^ : 
X-i — u, xi — v}, we have 

,. \{0<s<t:X^i{s)=u,Xi{s) = v}\ 

hm = 7r_i,i(u,w) 

t— >-oo t 

for all u,v G {0, 1}, where | • | denotes Lebesgue measure. 

Now let us consider an interval [to,ti) such that X_i(s) = u and Xi{s) — v 
for all s G [to,ti). Conditional on this, {Xq{s)} is a continuous-time Markov 
chain in {0, 1} that jumps from to 1 at rate P2u+v and from 1 to at rate 
q2u+v We can augment this process with fictitious jumps as follows: Consider 
the discrete-time Markov chain {Y{n)} in {0, 1} with one-step transition matrix 

/oo\ I <l2u+v P2u+v 

\ <l2u+v P2u+v 

and let {N{t)} be an independent, rate 1, Poisson process. Then {Y{N{s))} 
is the continuous-time Markov chain {Xo(s)} with fictitious jumps whenever 
player wins as a winner or loses as a loser. It allows us to determine the 
cumulative profit of player over the course of the time interval [tq,ti). The 
cumulative profit process has the following form: Let Sn — Sn-i be ±1 depending 
on the nth jump of {Y{n)}. (A jump to 0, from either or 1, means a profit 
of —1, while a jump to 1, from either or 1, means a profit of 1.) Then 
n^^Sn — >■ P2u+v — <l2u+v a.s. by the i.i.d. SLLN, so 

(34) —^ = — ^ — > P2U+V ~ q2u+v a.s. 

This assumes that the interval [tq, ti) on which X_i{s) — u and Xi{s) = v is 
the interval [0,oo). Of course, it is not. The set of times s for which X_i{s) — u 
and Xi{s) = w is a union of random intervals of the form [tq, ti), which we may 
write as 

[to, Ti) U [t2, T3) U [t4, T5) U • • • , 

where < tq < ti < T2 < • • • . We claim that, if these intervals are shifted to 
the left so as to fill up the interval [0, 00), then the profit process behaves as in 
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( 34 ) . This is due to the lack-of- memory property of the exponential distribution 



and to the fact that the two rows of ( 33 ) are equal 



Now we consider the cumulative profit process for player 0. Let St be cu- 
mulative profit by time t, and for u,v £ {0, 1} let S*}"'" be cumulative profit 
by time t accumulated at times s G [0, t) for which X_i(s) = u and Xi{s) — v. 
Then 

A 1 1 AIm,!") 



EE\ 



t 

u=Ou=0 
1 1 



EE 



|{0 < s < t : X_i(s) = u, Xi{s) = v}\ 



t 

u=Ov=0 



g{u,v) 



\{0<s<t:X^i{s) = u,Xi{s)=v}\ 
1 1 

= 2[^_i^i(0,0)po + 7r_ia(0, l)pi + ^„ia(l,0)p2 + ^-i,i(l, 1)P3] - 1 
= II a.s., 



^^T^-l,l{u,v){p2u+v - q2u+v) 



as claimed. We can now deduce ( [32| ) from ^. D 

Next we generalize Corollary [6] to our spin system. 



Corollary 14. Let us refer to the spin system on Z with flip rates (16) as 



the spin system with parameter vector (po7?'i;P2,P3), and let us denote fi of 



Theorem 13 by liipo,Pi,P2,P3) to emphasize its dependence on the parameter 
vector. 

(a) // the spin system with parameter vector ipo,Pi,P2,P3) is ergodic, then 
the spin system with parameter vector (93,^2, 91, ^o) is ergodic, and 

Kpo,Pi,P2,P3) = -^(93,92,91,90)- 

In particular, if po +^3 = 1 and pi +p2 — 1, then lJ,(j>o,Pi,P2,P3) = 0. 

(b) If the spin system with parameter vector (po,Pi,P2,P3) is ergodic, then 

2mm{po,pi,p2,P3) - 1 < h{pq,Pi,P2,P3) < 2max(po,Pi,P2,P3) - 1- 

Remark. Three of the four regions in Theorem [9] are invariant under the trans- 
formation A{pq,pi,P2,P3) :— (93,92,91,90), but region C corresponding to co- 
alescing duality is not. When pi = P2, we find that the (three-dimensional) 
volume of C U A(C) is 251/720, which is nearly 60% larger than the volume 
of C itself. However, the volume of the union of the five regions (the four re- 
gions of Theoremloland the transformed region C) is only 557/672, representing 
about 82.9% of the parameter space. When we do not assume that pi = P2, the 
(four-dimensional) volume of the union of the five regions can be estimated to 
represent about 79.3% of the parameter space. 
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Proof. Define the function -q : {0, 1}^ H^ {0, 1}^ by ri{x) := (. . . , 1 - x^i, 1 
Xo, 1 — xi, . . .). Let {X{t)} be the spin system on Z with initial state x and flip 



rates cf° '^^'^^'^^(a;) given by (16 1, where the superscripts are merely intended to 
emphasize the parameter vector. Then X'{t) := r]{X{t)) defines a spin system 
{X'{t)} on Z with initial state x' := ri{x) and flip rates cf '''='''^''°(a;'). To 
justify this assertion, it suffices to check that 

hence that ifPo,Pi,P2,P3(j- o 77) = (^93, 92, 91, 90 j) „ ,y. The first conclusion of 
part (a) follows. We can now take limits in Corollary [6] to obtain the second 
conclusion of part (a) as well as the conclusion of part (b) . D 

8 The case pQ = 1, p^ = for the spin system 

Suppose po — I and p^ — 0. Let 5 be the countable subset of {0, l}'^ consisting 
of states with alternating Os and Is with the single exception of a pair of adjacent 
Os or a pair of adjacent Is. In the special case in which pi = P2 = 1/2, if the 



initial distribution is concentrated on S, the spin system on Z with flip rates ( 16 1 
coincides with one studied in Example VIIL1.48 of Liggett (1985) in connection 
with the exclusion process. To justify this assertion, observe that states with 
three or more consecutive Os or three or more consecutive Is are inaccessible by 
our spin system, and these are the only states at which the flip rates differ in 
the two spin systems. 

If we assume that pi — p2 G (1/2, 1), then we can conflrm the Parrondo 
effect for our spin system, at least if the initial distribution is concentrated on 
S and if we consider the profit to a finite even number of consecutive players. 

Theorem 15. Let po — I, Pi — P2 & (1/2, 1), and p^ — 0. Let fig (resp., fi^ ) 
denote the mean profit per player per unit of time to an ensemble of N > 1 
consecutive players always playing game B (resp., C) in the spin system on Z 



with flip rates (16), assuming the initial distribution is concentrated on S. Then 



fig = for all even N > 2, fig does not exist for all odd N > 1, and fi^ > 
for all N > 1. In particular, the Parrondo effect is present if and only if N is 
even. 

Remark, (i) The quantities fig and fi^ have different meanings from those in 
Theorem [Tj They now represent mean profit per player per unit of time in the 
spin system model, whereas before they represented mean profit per turn (not 
per player) in the discrete-time Markov chain model. Notice that the Parrondo 
effect fails for odd N for a different reason here than it does in Theorem [T] 

(a) Under the assumptions of the theorem, the model has the following inter- 
pretation (stated in such a way that it can be generalized to higher dimensions): 
Voters are located at the sites of Z. Each has one of two possible positions on a 
particular issue. At exponential rate 1, each voter surveys his nearest neighbors 
and if there is a clear majority position on the issue among them, he adopts 
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the opposite position. If, however, there is an even spht on the issue among his 
nearest neighbors, he tosses a biased coin to determine his position. We might 
refer to the resuhing spin system as the biased contrarian voter model. 

We will need two lemmas in the proof. We state them here to avoid inter- 
rupting the argument. 

Lemma 16. Let the random variable T assume values in the set of even positive 
integers with f(n) := P(r = n) satisfying f{n) ~ cn"*^ as n —> oo through the 
even integers, for some constants c > and (3 € (1,2). Let Ai,A2,... be an 
independent sequence of exponential random variables with the odd-numbered 
terms having parameter Ai > and the even-numbered terms having parameter 
A2 > 0, and let {N(t)} be a rate 1 Poisson process. Assume that T, {A„}, 
and {N{t)} are independent, and define X := iV(Ai + A2 + • • • + At). Then 
P(X > fc) ~ Ck~P^^ as k —>■ 00 for some constant C > 0. 

Proof. Letting {Nj{t)} be independent rate 1 Poisson processes {j > 1), we find 
by conditioning that 

P{X >k\T^2n) 

= P(iV(Ai + A2 + • • • + A2„) > k) 

, 2n 

= P ( 5][7V(Ai + • • • + A,) - iV(Ai + • • • + A,_i)] > fc) 

= p('^[A^2j_i(A2,-i)+A^2j(A2j)]>fcY fc>0, n>l. 

Now if A is exponential with parameter A and independent of the rate 1 Poisson 
process {iV(i)}, we have 

00 ,m 



P(N(A) = m) = / — e-*Ae-^* dt 
Jo ml 



^ ^"i™e-(i+^)*dt=f^V^) , TO>0, 



to! ./o V 1 + A / V 1 + A 



which is (nonnegative) geometric(A/(l + A)). We conclude that 

00 00 

P{X >k)^Y^ f{2n)P{X >k\T = 2n)^Y^ /(2n)P(5'„ > k), 



n=l 



where S'„ := Xi + X2 + • • • + X„ and Xi,X2, . . . are i.i.d. with common dis- 
tribution equal to that of the sum of two independent random variables, one 
geometric(Ai/(l -I- Ai)) and the other geometric(A2/(l -I- A2)). In particular, Xi 
has mean and variance 



— + — and a := — ^ + — ^ 

Al A2 Ai A2 
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Let ak := fc^, where 76 (1/2, 1), and observe that 

< Y, /(2n)P(5„ > fc) < P(^LM-fc-«.J >k)+ J2 /(2n). 

n>l n>[fj.^^k — aki 

We can estimate these probabilities using Chebyshev's inequality: 

P('S'rp-ife+afc] > k) 

= 1 - P(S'p^-ife+a^] - r^^^/s + ak']n< k- Ifi^'^k + ak]n) 

> 1 - P(|5'p^-ife+a^^ - [M^^fc + "fclA^I > \k- \n^^k + ak^|J.\) 

^ _ g^rA*~^fc + «fc1 
~ (fc- [^-ifc + afe]^)2 

= l-0(fc-27+i) = i_o(l), 
and, if also 7 > /3/2, 

P(5'L^-iA;-afcJ > fc) 

= P(5L^-ife_Qj^j - LM^^fc - afcjA* > ^ - ll^^^k - ak\^i) 
< P(|5'L^-iA;-afcj - [fJ-^^k - ttfej^l > |fc - [f^^^k - ak\fJ.\) 

~ (fc- [^"ifc - afej^i)^ 
Finally, using 



/?-l 



- / (22:)- 


-^dx 


< > ; (2n)- 

n>7n 


/•OO 

Jm-l 



{2x)-^dx= ^-j-^ , 



we find that, given e > 0, we have for sufficiently large k, 

(^ - l)fc-^+i (^ - "^^)) 

< ^^^^T^ < oil) + (^-i)fc-m ' 

and our conclusion follows with C := c2^^ /j.^^^^ /{/S — 1). D 

Lemma 17. Let Xi,X2, . . . and X[,X2, . . . be two sequences of random vari- 
ables, all of which are nonnegative and i.i.d., with P(Xi > t) ^ Ct^^'"^ as 
i — >■ 00 for some C > 0. For each n > 1, let Sn '■= Xi + • • • + Xn and 
3',^ := X[ + ■ ■ ■ + X'^. Then = hminf„^oo Sn/S'^ < hmsup„^oo Sn/S'^ = 00 
a.s. 
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Remark. We do not know whether the assumption that P(Xi >t)^Ct ^'"^ as 
i — ?> oo for some C > can be weakened to -E[^i] = oo. 

Proof. First, by Kolmogorov's 0-1 law, there is a constant A such that 
hmsup„_j.gc Sn/S'„ = A a.s. It follows that liminf„_j.oo Sn/S'„ — \/A a.s., where 
l/oo := 0. Therefore, ^ > 1 and it suffices to prove that A = oo. 

The distribution of Xi is in the domain of normal attraction of the one- 
sided stable law of index a = 1/2 by the concluding remark of Feller (1971, p. 
581). This means that Sn/ri^ converges in distribution to the one-sided stable 
law of index a — 1/2. (When the index a is less than 1, there is no need 
for centering.) Since Xi > 0, all that is needed to verify this is to show that 
t^/'^Y'{Xi > t) ^- C as t ^- oo, which is what we assumed. (Here p = 1 and 
g = in Feller's formulation.) The concluding remark ultimately depends on 
Feller's (1971, p. 578) Corollary 2, but notice that "exponent a" in its statement 
should read "exponent —a". 

Now the limit distribution has a positive density by Lemma 4.1 of Kanter 
(1975). So we have S^/n^ ~>d V and SJn"^ -^d V , with V and V i.i.d. 
Hence [Sn/n'^S'Jn'') ->rf {V,V'), implying 5„/5; ^d V/V. But V/V also 
has a positive density, so given K > 1, ^^re have limsup„_^oQ P(S'n/S'^ > K) > 
V{V/V' > K) > 0. Hence P(S'„/S'; > K i.o.) = P(limsup„^^{S'„/S'; > 
K}) > limsup„^ooP('S'n/'S','j > i^) > 0. By Kolmogorov's 0-1 law, we have 
P{Sn/S'^ > K i.o.) = 1, hence P {Mm sup „_^^ Sn/S'„ > K) = 1. Since K was 
arbitrary, A = oo. D 

Proof of Theorem \lc\ First, we attempt to compute /i;^ assuming only pq — 1, 
Pi = P2 e (0, 1), and p3 = 0. 

There are two absorbing states, the two states with alternating Gs and Is. 
Motivated by the proof of Theorem [7J let us assume that the initial distribution 
is concentrated on S, defined at the beginning of this section. Let us label the 
states of S as follows. A state with a pair of adjacent Os is labeled by (0,i) 
if the leftmost of the pair occurs at site i. A state with a pair of adjacent 
Is is labeled by (l,i) if the leftmost 1 of the pair occurs at site i. Then the 
spin system is a countable-state continuous-time Markov chain in S with the 
following transition rates. From (0,i) the process jumps to (l,i -I- 1) at rate pi 
and to (l,i — 1) at rate P2- From (l,i) it jumps to (0,i + 1) at rate (j2 and 
to (0,1 — 1) at rate qi. There are two closed irreducible sets, 5° consisting of 
states of the form (0, even) and (1, odd) and S^ consisting of states of the form 
(0, odd) and (1, even). The process behaves as a simple random walk with jump 
rates that are periodic with period 2. From even states (i.e., those whose second 
component is even) in S^ and odd states in S^ it jumps to the right at rate pi 
and to the left at rate p2', from odd states in S^ and even states in S^ it jumps 
to the right at rate q2 and to the left at rate qi. If pi = P2, then qi = q2 and 
the walk is a simple symmetric random walk, but with jump rates 2pi at even 
states in 5" and odd states in S^ and 2qi at odd states in 5*° and even states 
in S^. (The notion of periodicity here is different from the one in the setting 
of discrete-time Markov chains; it merely means that the jump rates for our 
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simple symmetric random walk are not spatially homogeneous but depend on 
the parity of the state.) 

Let us consider what happens to players l,2,...,N with N even. Odd- 
numbered players will win every game they play and even-numbered players will 
lose every game they play if (a;o,a;i, . . . ,xn,xn+i) = (0, 1, . . . , 0, 1), hence if the 
walk starts in S'^ and its second component i satisfies i > N + I, or if it starts 
in S^ and i < — 1. Odd- numbered players will lose every game they play and 
even-numbered players will win every game they play if (xq, xi, . . . , x^, xat+i) = 
(1,0,..., 1, 0), hence if the walk starts in S^ and i > iV -I- 1 or if it starts in 
S° and I < — 1. The result is that, on excursions of the random walk away 
from 0, 1, . . . , A^, either odd-numbered players among 1, 2, . . . , A'^ will win every 
game they play and even-numbered players will lose every game they play, or 
the opposite will occur. In any case the sequence of wins and losses to players 
1,2, ... ,N during one of these excursions can be modeled as a compound Poisson 
process. Indeed, let 5'„ := ^i + • • • + ^„, where Ci, ^2, • • • are i.i.d. with P(^i = 
1) = P(^i = — 1) == 1/2, and let N{t) be an independent, rate TV, Poisson 
process. Then S'jv(i) represents the cumulative profit process to the ensemble of 
N players, since each player plays at rate 1. It follows that limt^oo t^^'S'jv(t) = 
a.s. To conclude that fj,g — 0, it suffices to show that the error incurred by using 
i~^5'jv(t) instead of the actual t~^St is negligible in an almost sure sense. The 
error is bounded by 2t^^ times the number of jumps by the Poisson process 
up to time t and during the time in which the continuous-time random walk is 
visiting {0, 1, ... , N}. If we denote the set of such times by It, then our bound 
is 

2t^ (number of jumps by {N{t)} during It) 

number of jumps by {N{t)} during It \It\ 

= m ~- 

The first fraction converges to N a.s., while the second tends to a.s. The latter 
assertion follows from Kesten's (1965) law of the iterated logarithm for the local 
time of simple symmetric random walk in discrete time. Let L{n, i) := |{1 < fc < 
n : Sk = i}\ and L{n) = sup^^^ L{n,i). Then limsup^^^^ L{n)/^ 2n log log n = 
1 a.s. The analogous result holds for simple symmetric random walks in contin- 
uous time by the SLLN for i.i.d. exponential random variables, and this suffices 
for our simple symmetric random walk with alternating jump rates, since the 
ratio max(pi, qi)/ nim{pi, qi) is finite. We conclude finally that fig — 0. 

Next we consider the case of N odd. If also N>3,\etSt'"' ^ and S^ 
denote the cumulative profit per player to players 1, . . . , A^ — 1 and to player N, 
up to time t. The previous paragraph tells us that limt_>.oo t~^St '""' ^ =0 a.s. 
It is therefore enough to consider \iuit~^oot~^St ■ Equivalently, we can assume 
without loss of generality that A^ = 1, so we focus on player 1. 

Player 1 will win every game he plays if (xq, xi, X2) ~ (0, 1, 0), hence if the 
walk starts in 5*° and its second component i satisfies z > 2, or if it starts in 
S^ and i < —1; player 1 will lose every game he plays if (xo,a:;i,X2) = (1,0, 1), 
hence if the walk starts in S^ and z > 2 or if it starts in S*" and i < —I. 
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The result is that, on excursions of the random walk away from {0, 1}, either 
player 1 will win every game he plays or player 1 will lose every game he plays. 
Define = tq < ^i < T2 < T3 < • • • in terms of the discrete-time simple 
symmetric random walk as follows, ti is the first time the walk visits 2; T2 
is the next time the walk visits —2; t^ is the next time the walk visits 2; T4 
is the next time the walk visits —2; and so on. Then Tk := Tk — t^-i are 
independent for A: > 1 and identically distributed for fc > 2, assuming values in 
the set of even positive integers; their well-known (Feller 1968, p. 89) distribution 
f{n) := P(T2 = n) satisfies f(n) ~ cn~^'^ as n — ?► 00 through the even integers 
for some c > 0, so Lemma [16] applies. To get player I's profit, we must convert 
these discrete-time excursions to continuous time and then run a Poisson process 
with rate 1 over those intervals, alternating between always winning and always 
losing. With St denoting the cumulative profit to player 1 up to time t, we 
claim that —1 — liminft_>.oo ^"^5** < limsupj_j.gc i^^Sf = 1 a.s., so that, with 
probability 1, the limit does not exist and hence /i^ does not exist. To prove 
this we need only consider a subsequence of times that achieves these limits. 
Using the result of Kesten described above, we can assume without loss of 
generality that on [Ti,r2), [T3,r4), and so on, player 1 always wins, while on 
['''2,T^), [T4,r5), and so on, player 1 always loses. (This assumes the walk starts 
in 5"°; if it starts in 5'^, the opposite occurs.) We also assume, again without 
loss of generality, that ri = tq = 0, that is, we ignore the time before the initial 
excursion. Let Xi,X2,... be the profits of player 1 accumulated during the 
winning excursions, and let X'l.X!^,... be the losses of player 1 accumulated 
during the losing excursions. The two sequences of random variables are i.i.d. 



and distributed as X in Lemma 16 with /3 = 3/2, Ai = 2pi, and A2 = 2qi. 
Denote partial sums hy Sn '■= Xi + ■ ■ ■ + X„ and S'^ :— X[ + ■ ■ ■ + X'^ for each 
n > 1. Then, along a particular subsequence of times, t~^St is asymptotic to 

Xi — Xi + X2 — X2 + • • • + Xn — X^ ^ Sn — Sn _ Sn/ Sn — 1 
Xi+X[+X2+X'^ + ---+Xn+ X; " Sn + S'n " Sn/S'n + 1 ' 

By Lemma |l7[ this has limit inferior equal to —1 a.s. and limit superior equal 
to 1 a.s., as required. 

This proves the assertions about /i;^. It remains to show that /i^ > when 

Pi =P2e (1/2,1) 

Now iJL^ = //(3/4, (1/2 -hpi)/2, (1/2 -hpi)/2, 1/4), which by Corollary 
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IS 



at pi = 1/2. If we could show that this function is increasing in pi, the proof 
would be complete. However, this monotonicity property appears difficult to 
prove. As in the proof of Theorem [7J we take an alternative approach. 

Let 7r_i^i be the —1, 1 two-dimensional marginal of the stationary distribu- 
tion -K when the probability parameters are 3/4, (l/2+pi)/2, (l/2+pi)/2, and 



1/4. We apply Theorem [13| twice. By Q and Corollary [12 

(35) ii'^ = ^o(l) ~ vro(O) - ^_i(l) - ^_i(0) 

= ^-i,i(l, 0) + ^-ia(l, 1) - [^-i,i(0,0) + 7r_i^i(0, 1)] 
= ^_i,i(l,l)-^_ia(0,0). 
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By ^, Corollary [12) and ([35]), 

fi^ = ^_ia(0,0)(l/2) + ^_ia(0, l)(2pi - 1) + 7r_i,i(l, l)(-l/2) 



N 



= (2pi-l)7r_ia(0,l)-(l/2V;^ 
Therefore, /i^ = (2/3)(2pi - l)7r_i,i(0, 1). 



We claim that 7r_i.i(0,l) > 0. Suppose not. Then, by Corollary 12 



7r_i_i(0, 1) = 7r_i^i(l,0) = 0, so by Lemma 11 it is concentrated on the four 
states that have the same spin at every odd site and the same spin at every even 
site. Let us denote those four states by (all Os), 1 (all Is), lovon (Is at even sites 
only), and lodd (Is at odd sites only). Then there exist nonnegative ao, oi, 02, 03 
with ao + ai + 02 + 03 ~ 1 such that tt — aoSo + aiSi + a2Si^^^^ + a^Si^^^. 
But this leads to a contradiction. We apply J J^fdn = with f{x) :— xq, 
f{x) := cc-iXi, and f{x) := a;oX2, to get four linear equations in ag, ai, 02, 
and 03, namely Sao ~ 3ai ~ a2 + a^ — 0, — 6ai — 2a3 — 0, — 6ai — 2a2 — 0, and 
ao + ai +a2 + 03 = 1, for which there is no solution with oq, ai, 02, 03 > 0. Since 
Pi > 1/2, we conclude that fi^ > 0. D 

9 Open problems 

Suppose, for example, that (po,Pi,P2,P3) = (1/10,3/5,3/5,3/4). Then the spin 
system on Z with flip rates ( [l6| is attractive, so by Theoremloj it is ergodic. (In 
fact, conditions (c) and (d) of Theorem lo] also apply.) Hence by Theorem 10 we 
know that limjv_i.oo fJ-B ^^'^ liniAr_>.oo fJ,Q exist. We have computed these means 
exactly for 3 < A^ < 19 (Ethier and Lee 2012) and have found that the former 
has stabilized to 4 significant digits by A^ = 19, while the latter has stabilized to 
11 significant digits by TV = 19. In fact, in both cases, the 17 computed numbers 
are monotonically increasing (why?). Also, /i^ < —1/500 for 3 < A^ < 19 and 
H^ > 1/100 for 4 < A^ < 19. Can we say anything about the limits, based on 
these computations? Can we conclude that the Parrondo effect is present for 
all N > AI Or that it is present in the spin system? 

The monotonicity problem mentioned in the proof of Theorem [7] is in- 
teresting. More generally, assuming that the Markov chain is ergodic, let 
fJ''^iPo,Pi,P2,P3) be the mean parameter of Theorem [5] Is this function mono- 
tonically increasing in each variable? A partial result in this direction is that it 
is nondecreasing in each variable if we restrict to parameter vectors satisfying 
the attractiveness condition, po < imn{pi,p2) < max(pi,p2) < Ps- (This fol- 
lows from Theorem III. 1.5 of Liggett 1985.) Of course, the same question can 
be asked about the limiting mean function. 

Can ergodicity of the spin system be established more generally? For exam- 
ple, is it sufficient that Pq:Pi:P2:P3 G (0, 1)? It seems likely that Theoreml9lcan 
be strengthened, perhaps via another form of duality. 

We have considered only the one-dimensional integer lattice. Mihailovic and 
Rajkovic (2006) studied the two-dimensional MA^-player model with periodic 
boundary conditions, assuming that the bias of the coin tossed depends only on 
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the number (not the set) of winners among the four nearest neighbors. Using 
computer simulation they found evidence of the Parrondo effect for parameters 
close to those of the voter model. What can be proved in dimensions greater 
than one? 
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